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What do we want? 

What is the value of the Workmanship Standards? 






[Workmanship] Quality Assurance: 


• Reduces occurrences 
of defects 

Defects that prevent 
achieving: 

- intended design 

- “durability” goals (reliability) 


Lunar 

Reconnaissance 


Orbiter (LRO) 




LRO Processor Board 









[Workmanship] Quality Assurance: 

• Quality Baseline: “Best we know how” to build 
quality in at low level of assembly 


- Screening for observable artifacts shown in the past to cause failure 
early in life (infant mortality) or when exposed to vibration, shock, and 
thermal cycling (“qualification”). 


- Guides (requires) manufacturers to processes known: 

• to produce highly repeatable interconnects which haven't failed 

• to reduce un-screenable defects (latent defects) 







[Workmanship] Quality Assurance: 


• Ensures applied item is 
as good as qualified 
item 

- Process repeatability 
provides traceability to 
qualification 



Thermal cycling of partially-filled solder joints 






Where do Workmanship defects come from? 

* Unsuccessful design 

* Low manufacturability 

* Low-quality building blocks 

(parts and materials) 

* Under-developed or 
uncontrolled processes 

* Untrained personnel 

* Screening escapes Cracks ion surface of printed circuit board (PCB) 








How do we ask for absence of defects due to 
poor engineering design? 

From J-STD-ooiES, 1.13.2 

The development [of acceptance 
criteria for nonstandard 
configurations] should include user 
involvement. The acceptance criteria 
shall have user agreement. Mounting 
and soldering requirements for 
specialized processes and/or 
technologies not specified herein 
shall be performed in accordance 
with documented procedures which 
are available for review. 



Excessive heat rise created in non-standard interconnect 




Printed Circuit Board coupon analysis + respinning non-compliant lots 
revealed this design to have very low manufacturability 







How do we ask for absence of defects due to 
poor engineering design? 

From NASA-STD-8739.4, para 7.1 

Precautions shall be taken to prevent the 
mismating of connectors, caused by interchanging 
or by reversing, through one of the following 
techniques: 

7.1.1 Use of constraints that locate similar 
connectors built into interconnecting cables and 
harnesses so they cannot be interchanged 
(Requirement). 

7.1.2 Selection of different sizes for connectors to 
be located adjacent to each other (Requirement). 

7.1.3 Polarization or dissimilar keying of adjacent, 
similar connectors (Requirement). 





How do we ask for absence of defects due to 
poor engineering design? 



From NASA-STD-8739.1A 


9.1.4 Mandatory Staking. 


9.1.4.1 All required staking shall be detailed on 
the engineering documentation (Requirement). 


9.1.4. 2 Jumper wires in excess of 2.34 cm (1 
inch) and axial leaded tantalum capacitors of all 
case sizes shall be staked. 


9.1.4.3 If parts are identified to be staked but 
staking location or staking dimensions are not 
specified on the engineering documentation, 
the following shall be used as default criteria... 



How do we ask for absence of defects due to 
low quality parts, materials and processes? 

From J-STD-ooiES, 3.1 

Materials The materials and processes used to 
assemble/manufacture electronic assemblies shall be selected 
such that their use, in combination, produce products acceptable 
to this standard. 

• Solder and flux defined, shelf life 

• Prep required: cleaning, baking, tinning, gold 
removal 

• Fixturing 

• Tool calibration 

• ESD Control 

• Thermal shock protection 

• Storage and handling 




How do we ask for absence of defects due to 
uncontrolled processes? 

From J-STD-ooiES, 4.17 

The Manufacturer shall develop and maintain operating procedures describing the 
reflow soldering process and the proper operation of the equipment. These procedures 
shall include, as a minimum, a reproducible time/temperature envelope including the 
flux and solder paste application procedures and coverage, drying/degassing operation 
(when required), preheating operation, controlled atmosphere (if used), solder reflow 
operation, and a cooling operation (see 4.15.2). These steps may be part of an integral or 
in-line system or may be accomplished through a series of separate operations. When 
PCAs are required to be subjected to additional mass reflows in excess of the 
documented manufacturing process plan, the reason for the additional processing shall 
be documented, and notification shall be provided to the User within 24 hours. 

From J-STD-ooiES, 3.1 

When major elements of the proven processes are changed (e.g., flux, solder paste, 
cleaning media or system, solder alloy or soldering system), validation of the 
acceptability of the change(s) shall be performed and documented in accordance with 
approved tests agreed upon between the Manufacturer and User. The change shall be 
approved by the User prior to use. Major elements may also pertain to a change in bare 
boards (including supplier), solder resist, or metallization. 


How do we ask for absence of defects due to 
uncontrolled processes? 

From NASA-STD-8739.4, para. 5.1.2 

The supplier shall assure that the design personnel are familiar with the 
requirements of this Standard, crimping, cabling, and harnessing techniques, and 
other pertinent requirements of the contract (Requirement). The supplier shall 
implement and document a training program which provides the necessary 
training of fabrication and inspection personnel in crimping, cabling, and 
harnessing requirements and techniques (Requirement). Use of equipment and 
procedures pertinent to their responsibilities in performance of the contract 
requirements shall also be documented (Requirement). The supplier is 
responsible for certifying and maintaining the certification of each individual who 
fabricates, inspects, or instructs. 

Commitment to using known good designs, materials, parts, processes, and trained 
personnel also streamlines failure root cause analysis and continuous improvement 




How do we ask for defect screening? 



• Inspection conditions defined 

• Accept/reject criteria defined 

• Operators and inspectors trained 


Most well known and understood 
parts of the Workmanship quality 
approach. 





Commercial-off-the-shelf (COTS) 

NASA-STD-8739.6, para. 1.2.3, “The workmanship requirements of [NASA’s 
Workmanship Standards] do not apply to suppliers of commercial-off-the-shelf 
(COTS) items. Projects which use COTS hardware for applications described 
in 1.2.2 above are responsible for identifying and managing risk associated with 
hardware that was built without material controls, production methods, and/or 



Source: www.gore.com 




We know what we want 
We have a way to ask for it 
Do we always get it? 

No. 




Why don't we always get compliance to 
Workmanship requirements? 

Failure to flow down requirements. The requirements are there but they do not 
get applied or they are not flown down to subcontracts. WHY? 

• Hardware is “off-the-shelf” 

• From ‘heritage’ NASA procurements which pre-date assurance 
requirements like Workmanship (includes subcontractors) 

• Leveraging off of DoD, aeronautics, automobile, or commercial market 
where Workmanship requirements may not be uniformly applied. 

• Supplier does not have the sophistication or volume to establish standard, 
qualified manufacturing processes 

• University or other R&D interest 

• Niche manufacturer; single source supplying item with unique performance 

Increasing absence of process engineering and packaging qualification testing. 

• Dependence on design + fabrication using old or existing techniques. 

• Destruct units are too expensive. 




Do our contract technical representatives, 
designers, and quality engineers understand the 
difference between Performance and 
Assurance? 


Grounding through a 
dissipative surface for ESD 
control shall provide 
connection resistance 
between io 6 Q and io 9 Q 


Ground connections used for 
ESD Control shall be verified 
through monthly 
measurements and recorded. 


How do you know it will work? 



How will it work? 



Do we mistake craft for engineered solutions? 


“We always use rosin type 
flux because it works 
well.’' 

If everyone knows “the best way”, 
their understanding of it is identical, 
materials and tools remain unchanged, 
and the approach is always used, then 
performance is achieved. (Luck vs. 
Assurance) 


“Only rosin flux of category 
type ROLo or ROLi, as 
defined by IPC-STD-004, is 
permitted.” 

Demands “the best way” using a 
specific, quantitative, and 
repeatable interpretation of it 
(performance and assurance). 



Note: Training is an assurance method. 



Path to Success: Luck or Quality Assurance 


Experienced Operators 

Process Used Before 


Process is Documented 

Process Parameters Controlled 


Luck avoids: 

Design 
requires 
different or 
new 

manufac- 

turing 

techniques 


Produced items 
are not identical 
creating 
opportunity for 
false positive in 
reliability testing 


Process may 
not be 

repeatable or 
known by 
everyone 

Process is 
operator- 
dependent and 
that operator is 
not available 


Product is Qualified 


No evidence 
that design is 
reliable 


Begin to see 
repeatable 
predictable 
outcomes 


Operators are Trained 


Untrained Defect Screening 

operators = 

uncontrolled 

variable 


Representative 
Item is proven 
to be capable 
of application 


Defective items 
removed from 
population to be 
used in mission 


Assurance ensures »»» 
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We want to use this particular item but the 
supplier doesn’t train their operators to the 
Workmanship standards. What’s the risk? 


35 

30 




This is non-standard . Can you inspect this for 
Workmanship and see if it is OK? 















Our supplier doesn't use rosin flux. Is that OK? 




What is the risk associated with “use as is” when an 
interconnect is found with a Workmanship defect ? What 
is sufficient evidence of risk mitigation ? 








Closing thoughts... . 


The Workmanship Standards serve to force positive outcomes 
rather than to depend only on finding and repairing defects. 

The Workmanship requirements address all avenues that can 
introduce defects or allow them to exist at the next level of 
integration: design, materials, processes, process control, 
screening. 

Though we value screening assemblies for defects, we prefer 
that defects are prevented. Repair is becoming less tolerable. 




Closing thoughts... . 


Defect prevention is highly dependent on Process Engineering though 
this role is not named in the Workmanship standards nor is training 
required. 

NASA’s business model draws them to some suppliers who are not 
positioned to comply with Workmanship requirements partially or in 
total. 

Attention to assurance vs. performance can help clarify risks 
associated with production lines which do not align with Workmanship 
requirements (What quality techniques are in place to keep defect rates very low?) 

Opportunities for requirements relief may exist when reliability 
engineering sorts through definitions of defects by relevant 
environment. 




